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ATTENTION DETECTION 

Field of the Invention 

5 

The present invention relates to detection of an attention state of at least 
one person, and particularly, although not exclusively, to a method and apparatus 
for autonomously collecting images. 

10 Background to the Invention 

Known camera technology, including cameras of the conventional film type, 
and digital cameras, consumer type cameras and professional type cameras, are 
based upon a usage model which requires a human photographer to point the 
15 camera at a field of view containing a subject of which a picture, image or video is 
to be captured, and activating a trigger button causing the camera to capture a 
still image, or in the case of a video camera, a moving picture scene. 

Known still image cameras and video/film cameras for consumer and 
20 professional use are effectively products which are used by a photographer to 
capture an image or moving sequence of a scene, where the photographer 
consciously selects the scene* sets up the camera positioning for a particular 
view, and activates capture of an image or sequence of images as a 
photographer controlled activity. 

25 ' . v 

In some cases, automatic triggers are provided on a timer device, whereby 
the photographer can set a camera up pointing at a scene, activate a time delay 
trigger, giving the photographer time to enter the scene, and then the camera 
automatically takes a picture or sequence once the pre-determined time limit has 
3 0 elapsed, allowing the photographer to appear in her own images. 
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There are also known prior art surveillance camera systems, which 
automatically capture still images or video sequences. Such cameras are used 
for security purposes in commercial premises such as factories and offices, as 
well as in domestic environments for security. These known cameras capture 
images or sequences of images in response to sensor activated events, such as 
an infra-red sensor being triggered, a movement sensor, or a circuit breaker 
sensor, for example a sensor which is activated when a door or window is 
opened. In general, this type of camera is wall mounted or mounted on posts, 
and pre-arranged to take a picture of a p re-determined scene. The cameras are 
not mobile and provide a permanent security monitoring installation. 

In some cases, surveillance cameras are directable remotely from a central 
console unit, so that they can pan across a field of view and focus in and out of 
that field of view by remote control, the cameras being moved and controlled by 
servomotors. 

There are also known surveillance cameras for other uses, for example 
traffic monitoring, and speed detection, which are triggered by motion sensors 
which detect vehicles moving within pre-set speed limits. This type of camera is 
conventionally installed in a permanent installation, pointing at a pre-determined . 
scene. 

The known surveillance cameras and traffic monitoring cameras provide 
fixed installations which operate without a human photographer, but are limited in 
their fields of view. 

More recently, compact cameras have been installed in personal 
communications devices, such as mobile phones. Pictures of individuals can be 
taken using hand held devices, and sent as messages over a wireless 
communications network to personal handsets of other users. With these hand 
held devices, the basic mode of usage requires a person to act as a 
photographer, pointing the device at a field of view, for example directing the 
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device at their own face to take a picture of themselves, and activating capture of 
an image by triggering a switch. 

The concept of a user wearable camera device receiving attention clues 
5 from a host wearer and capturing images is known. For example in Summarizing 
Wearable Video', K. Aizawa, K. I. Ishijima, M. Shiina, IEEE, International 
conference on image processing, 111:398-401, Thessaloniki, Greece, 2001, there 
is disclosed a system which receives sensor inputs from a host wearer, and 
which attempts to understand those sensor inputs in order to determine when to 
10 perform capture of an image. 

Other known work which analyses a host wearers attention from a self 
perspective point of view includes the following: 

15 J Healey and R.W. Picard, "StartleCam: A Cybernetic Wearable Camera", 

In Proceedings of the International Symposium on Wearable Computers, pages 
42-49, 1998. 

A.Lockerd, F. Mueller, "LAFcam - Leveraging Affective Feedback 
2 o Camcorder", ACM CHI, 2002. 

Y Nakamura, J.Ohde, Y. Otha, "Structuring personal activity records based 
on attention: Analysing vidoes from a head-mounted camera", in International 
Conference Pattern Recognition, Barcelona, September 2000. 

25 

Known usage models and systems for detecting a persons attention focus 
on analyzing the actions and behavior of a person from the perspective of that 
person, that is analyzing parameters which are local to a person and in an 
immediate vicinity of a person from a position at the host person. 



30 
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A further body of known work is concerned with observing a person from a 
position external of that person, that is from an external perspective. This known 
body of work includes the following items: 

APentland, "Looking at People: Sensing for Ubiquitous and Wearable 
Computing, IEEE Trans. Pattern Analysis and Machine Intelligence, IEEE CS 
Press, Los Alamitos, Calif., Jan. 2000, pp. 107-118. 

M.Trivedi, I.Mikic, S. Bhonsle, "Active Camera Networks and Sematic Event 
Databases for Intelligent Environments", IEEE Workshop on Human Modeling, 
Analysis and Synthesis, June 2000. 

APentland, T.Choudhury. "Face Recognition for Smart Environments", 
IEEE Computer, vol. 33, no. 2, February 2000, pp. 50-55. 

D.Gavrila, "The Visual Analysis of Human Movement: A Survey", Computer 
Vision and Image Understanding, vol. 73, no.1 , January 1999, pp. 82-98. 

V.Pavlovic, R. Sharma, T. Huang, "Visual Interpretation of Hand Gestures 
for Human-Computer Interaction: A Review". IEEE Trans. PAMI, vol. 19, no. 7, 
July 1997, pp. 677-695. 

R. Stiefelhagen, J.Yang, A.Waibel, "Estimating focus of attention based on 
gaze and sound", Proceedings of the Workshop on Perceptive User Interfaces, 
2001. 

R.Vertegaal, R.SIagter, G.van der Veer, A.Nijholt, "Why conversational 
agents should catch the eye", In Summary of ACM CHI Conference on Human 
Factors in Computing,, The Hague, 2000. 
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Some known devices such as disclosed in the above bodies of work could 
be referred to as 's\ which observe a person from a position external of that 
person. 

5 Summary of the Invention 

A first aspect of the present invention relates to autonomous image capture, 
and the detection of situations of interest to users of cameras, in order to trigger 
automated image capture without conscious human user activation of a camera 
10 device. Embodiment of the invention provide an autonomous wearable camera 
system which is able to capture images of moments by inferring situations of 
interest to persons. 

Further embodiments may provide for the use of an agent device which is 
15 able to interpret 'attention clues' given out by human persons, the attention clues 
being used to activate one or more image capture devices for capture of images 
when the attention clues are detected. The device may be implemented as one 
or more firmware devices, or as a software agent. Specific implementations 
herein aim to implement a system for autonomously determining a situational 
2 0 saliency based upon the attention clues, in order to trigger capture of one or a set 
of images in that situation. 

In one embodiment, a user carries a wearable camera. The agent device 
may be situated on the wearer's body. In other embodiments, the device may be 

2 5 situated elsewhere, or the agent device may be distributed. 

Where the agent device is worn by a person carrying a camera, the agent 
device may be capable of adopting a 'self-perspective' mode, in which the agent 
device observes situations from the perspective of a host person who is wearing 

3 0 an image capture device. In this mode, the agent device may receive attention 

clues in the form of sensor signals of parameters of the wearer, for example skin 
conductivity, body movement, or voice utterances made by the host wearer. In 
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the self-perspective mode, since the agent device is in close proximity to a 
person. In order to capture images in the self perspective mode, the agent 
device requires input from sensors strategically placed on the host wearer, 
including limb positioning sensors, eye trackers, and the like. 

The agent device combines information collected about the host wearer in 
the self perspective mode with externally received information collected about the 
same host wearer or similar or complimentary device which has collected 
information in an 'external' perspective mode. The agent device determines from 
the combination of self perspective attention clues and external perspective 
attention clues whether the host wearer is experiencing a situation in which their 
attention is raised to a level where recording of an environmental situation in 
which the user finds themselves would be of interest to that user. 

In an 'external' perspective mode, an agent device may analyze a another 
person's body language and attention clues from a perspective external to the 
person, that is, as an entity remote from the person and observing the person. 
The agent device may be physically positioned away from that other person, and 
may observe that other person. In this mode, the agent device may detect 
externally detectable attention clues given out by the other person, for example 
limb movements, pointing of a finger, movement of eye look direction, rapid re- 
orientation of a head of the person, aspects of the other persons body posture, 
and like attention indicating body language signals. 

According to one aspect of the present invention, there is provided an 
attention detection system comprising: 

a portable attention detection module capable of receiving attention clues 
of a host person, said attention clues generated from a self perspective of said 
host person, and receiving attention clues of said host person generated from an 
observer perspective; 



200207945 



-7- 

at least one sensor device capable of generating signals relating to said 
attention clue signals of said self perspective; and 

5 at least one sensor device capable of generating signals relating to said 

attention clue signals of said observer perspective.; 

wherein said attention detection module is capable of determining a 
10 situation of raised attention of said host wearer from said self perspective 
attention clues, and said received observer perspective attention clues. 

Other aspects of the invention are as recited in the claims herein. 

15 

Brief Description of the Drawings 

For a better understanding of the invention and to show how the same may 
be carried into effect, there will now be described by way of example only, 

2 0 specific embodiments, methods and processes according to the present 

invention with reference to the accompanying drawings in which: 

Fig. 1 illustrates schematically usage of one embodiment of a personal 
image capture device worn by a host person ; 

25 

Fig. 2 illustrates schematically operation of one implementation of an image 
capture system comprising a plurality of image capture devices, each worn by a 
corresponding host person 

3 0 Fig. 3 illustrates schematically logical entities comprising a specific 

implementation of an image capture system 
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Fig. 4 illustrates schematically information processing stages for processing 
information in the image capture system of Fig. 3; 

Fig. 5 illustrates schematically components of one implementation of a 
5 people observer device; 

Fig. 6 illustrates schematically components of one implementation of an 
attention detection module; 

io Fig. 7 illustrates schematically components of one implementation of an 

image capture device; 

Fig. 8 illustrates schematically an example of various sensor types 
according to the specific implementation of the image capture system; 

15 

Fig. 9 illustrates schematically a specific embodiment of an image capture 
system comprising three portable devices, each comprising an image capture 
device, a people observer device and an attention detection module; 

2 0 Fig. 10 illustrates schematically an analysis of a two dimensional captured 

digital image of a person according to a specific method 

Fig. 11 illustrates schematically a specific embodiment of an analyzer 
component of an attention detection module 

25 

Fig. 12 illustrates schematically sensor inputs to a people observation 
device in a self perspective mode of operation of an image capture system; and 

Fig. 13 illustrates schematically a specific method of interaction between 

3 0 first and second attention detection modules each operating in a self-perspective 

mode and in an observer mode. 
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Detailed Description of a Specific Mode for Carrying Out the Invention 

There will now be described by way of example a specific mode 
contemplated by the inventors for carrying out the invention. In the following 
5 description numerous specific details are set forth in order to provide a thorough 
understanding. It will be apparent however, to one skilled in the art, that the 
present invention may be practised without limitation to these specific details. In 
other instances, well known methods and structures have not been described in 
detail so as not to unnecessarily obscure the description. 

10 

Referring to Fig. 1 herein, there is schematically illustrated a person carrying 
a portable image capture device capable of capturing images of a scene in a field 
of view of the image capture device. Typically, the image capture device may 
comprise a digital camera having a charge coupled device (CCD) array. 

15 

The image capture device may be activated remotely by a wireless signal 
from a remote transmitter, in order to capture an image. The image capture 
device may also be activated by signals generated by local sensors worn on the 
persons body. 

20 

The portable image capture device comprises an attention detection module 
(ADM) which receives attention information indicating an attention level of the 
host wearer from a device external of the host wearer. 

2 5 The attention detection module communicates with at least one external 

people-observing device, such that the attention detection module can combine 
self perspective information received directly from the host wearers body, with 
observer perspective information concerning the host wearer, which is received 
from the external people-observing device, in order to determine a condition 

3 0 under which the host wearer is experiencing a situation in which the wearer's 

attention is raised, and in response to such a condition, for activating capture of a 
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still image, image sequence, or capture of another like media, from a host 
perspective. 

Whilst an image capture device is described herein, with reference to the 
first embodiment of Fig.1, attention clues of the host wearer can be used for 
capturing any other media type, for example including but not limited to audio 
data, still images, video sequences, pressure or temperature changes, or other 
changes in environmental parameters which can be experienced from the 
perspective of the host person. 

Referring to Fig. 2 herein there is schematically illustrated a situation of use 
of an image capture system in which 2 individual users are in proximity to each 
other, each user carrying a portable image capture device, which in this case 
incorporates an agent device capable of observing a host person wearing the 
image capture device, as well as observing other persons. The users are close 
enough such that an image capture device of a first user can capture a still or 
moving image of a second user, and are in close enough proximity that first and 
second image capture devices can communicate with each other over a wireless 
link, for sending and receiving activation signals for activating capture of an 
image. 

In this situation, the persons interact with each other using body language, 
such as waving, smiling, laughing, gesticulating, and adopting postures. Body 
language signals are collected through a system of sensors, which input attention 
signals into a (POD) comprising the agent device. The converts the attention 
signals into attention clue data. The attention clue data is processed to 
determine whether there is a situation in which the body language of the persons 
is indicating that there is enough interest to the persons, that capture of an image 
may be appropriate. If the saliency of the situation high enough that capture of 
an image is appropriate, then one or more image capture signals is generated for 
activating one or more image capture devices to capture one or more images. 
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ln the situation illustrated in Fig. 2 herein, a range of attention clues are 
collected, both from a self perspective, and from an external observer 
perspective, concerning the body language and behavior of the first host user. 
Self perspective attention clues, for example body movement, breathing rate, 
5 heart rate, blood pressure, skin resitivity, direction of motion and the like may be 
measured by sensors worn by the host wearer. However other attention clues 
which are available from an observer perspective, but which are difficult to assess 
from a self perspective are provided by the which is external of the host wearer of 
the image capture device. Such attention clue information may include 
10 parameters such as for example overall body posture, eye gaze direction, facial 
expression, overall body motion, or limb motion such as waving, pointing location 
of the host wearer with respective to other objects in an environment or the like 
which are best observed from a position external of a person. 

15 The attention detection module carried or worn by the host wearer 

assimilates both sets of attention information, that is, the self perspective and 
observer perspective attention clues, to determine a state of attention of the host 
wearer of the attention detection module. 

20 Referring to Fig. 3 herein, there is shown schematically in a general case 

logical entities present in an image capture system according to a specific 
implementation of the present invention. The logical entities comprise a plurality 
of s 300, 301 ; a plurality of attention detection modules 302, 303; a plurality of 
image capture devices 304, 305; and a plurality of sensors 306, 307. 

25 

In this example shown, each user carries a 300, 301; an attention detection 
module 302, 303; and an image capture device 304 to 305. However, in the 
general case, the location of the s, and attention detection modules need not be 
restricted to being worn by human persons. Similarly, although in many 
3 0 implementations sensors are worn by individual persons, in the general case, 
sensors can be provided at remote positions, for example mounted on walls or 
posts, for observing persons remotely. 



200207945 



-12- 

Each comprises a set of algorithms for detecting attention clues exhibited 
by a host wearer of the . Each receives signals from one or a plurality of sensors 
306, 307 respectively. 

5 

Each attention detection module comprises one or more algorithms for 
analyzing attention clues relating to the host wearer of a particular image capture 
device, and for analyzing attention clues received externally from other s relating 
to attention clues exhibited by one or more other host persons. 

10 

Referring to Fig. 4 herein, there is illustrated schematically information 
processing stages shown in the image capture system of Fig. 3 herein. 

Body language information 400 exhibited by the persons in proximity of the 
15 s is detected by one or more sensors. The sensors generate sensor signals, 
which comprise signals, sensing various parameters, for example movement, 
sound, 2-dimensional images and the like. The sensor signals are input into an 
attention detection module, which analyses the sensor signals to determine a set 
of attention clues collected from a self perspective. Attention clues comprise data 
20 which represents the information that the attention of the host wearer person has 
been raised. The attention clues are input into an attention detection module. 
The attention detection module also receives attention clues 403 generated by an 
external people-observing device relating to the host wearer person. The 
attention detection module processes the attention clue data to determine 

2 5 whether an attention level of the host wearer person, is at a level where an image 

of a situation is likely to be of interest, and worth capturing. If so, the attention 
detection module generates a set of camera trigger signals 404 for activating 
capture of images by one or more image capture devices. This results in a set of 
images 405 of a situation which has occurred which has raised an attention level 

3 0 of one or more persons to a level such that it triggers capture of the image. 
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Referring to Fig. 5 herein there is shown schematically components of a 
people-observing device 500. The people-observing device comprises a 
transponder 501 and an antenna 502; a controller 503 for controlling the device; 
optionally, an image capture device 504 for capturing images of scenes; an 
5 image memory 505 for storing image data of images captured by the image 
capture device; a proximity sensor 506 for sensing when the people-observing 
device is within range of one or more other people-observing devices and/or one 
or more attention detection modules; a host sensor interface 507 for receiving 
signals from a plurality of host sensors which may be worn by a host person 
10 wearing the people-observing device; a set of connectors 508 which can be used 
optionally for directly connecting one or more sensors to the host sensor 
interface; and an attention detector 509 for detecting attention clues of a host 
wearer, and/or one or more other persons. 

15 First and second people-observing devices may be provided with beacons 

to locate and detect each other, for enabling an exchange of information between 
people-observing devices. A said people-observing device may be configured for 
recognizing at least one other people-observing device to form a group of people- 
observing devices which recognize each other. The people-observing device 

20 may be configurable to communicate or not communicate with at least one other 
people-observing device, based on an analysis of contextual information relating 
to a host person, or an observed person. Such contextual information may 
comprise a location of a person, a time of day , a local environment in which the 
person finds themselves, or like contextual information. 

25 

The people-observing device may be configurable to communicate or not 
communicate with at least one other people-observing device based on an 
analysis of a type of attention clue detected, for example, whether a short casual 
glance is exchanged with a person, or whether a longer glance is exchanged with 
3 0 a person. 
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The people-observing device may also be configurable for being set to 
communicate or not communicate with at least one said attention detection 
module depending upon a type of attention clue detected, for example a 
particular facial expression, a body posture type, a head direction or an eye gaze 
direction or an arm gesture. 

Referring to Fig. 6 herein, there is shown schematically components of an 
attention detection module 600. The module comprises a transponder 601 and 
an antenna 602 for communicating with one or more other attention detection 
modules, and one or more people-observing devices; a controller 603; a host 
people-observing device interface 604 for interfacing with a people-observing 
device worn by a same host person as may be wearing the attention detection 
module; a self perspective attention analyzer 605 for analyzing attention clues of 
a host wearer; an observer perspective analyzer 606 for analyzing attention clues 
observed of persons other than the host wearer person; and an internal data bus 
607 connecting the above elements. It will be appreciated by persons skilled in 
the art that power supplies, casings, and like known components are also 
provided in order to create a practical manufactured unit. 

The attention analyzers comprise various components for detecting when 
the attention of a person or persons is raised. These components include a 
component for detecting a facial expression of one or more persons, a 
component for detecting an eye direction of a person, a component for detecting 
an attitude of movement of one or more persons, a component for detecting a 
body posture of one or more persons, and a component for detecting a context 
of an environment containing one or more persons. 

In the foregoing embodiments and implementations, an attention detection 
module has been shown worn personally by a host wearer. The attention 
detection module is personal to a host wearer, in the sense that an attention 
detection module monitors a host wearer and receives attention clues which have 
been collected from the perspective of the host wearer, and attention clues 
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concerning that host wearer which have been collected from an observer 
perspective from a position external of that host wearer. The physical location of 
the attention detection module is not restricted to being worn or personally carried 
by a host wearer, but may itself be remote from the host wearer. However, an 
5 attention detection module is assigned to a host wearer for detecting an attention 
status of that host wearer, and therefore in that sense is personal to that host 
wearer. However in a preferred embodiment, an attention detection module 
would be carried or worn by a host wearer. 

10 Referring to Fig. 7 herein, there is illustrated schematically components of 

an image capture device 700. The image capture device comprises a 
transponder 701 and an associated antenna 702; a set of image capture optics 
and sensors 703, for example a charged coupled device (CCD) array; a controller 
704; an image memory 705 for storing image data; an internal bus 706; and a 

15 people-observing device interface 707, for connecting to a local people-observing 
device, where this is worn by a same host user as the image capture device. 

The image capture device is capable of receiving trigger signals from one or 
more attention detection modules, over a wireless link using the antenna 702 and 
20 transponder 701. Activation of the image capture device causes the device to 
capture an image or image sequence and store this in the image memory 705 
under control of the controller 704. 

The image capture device is also capable of acting as a monitoring sensor 

2 5 for collecting 2-dimensional images of one or more persons, for input into a 

people-observing device. In the embodiments shown, the capture image data 
may be transmitted over the wireless link via the antenna 702 and transponder 
701 to a remote people-observing device. 

3 0 It will be appreciated by the person skilled in the art that although a people- 

observing device 500, attention detection module 600, and an image capture 
device 700 are illustrated in Figs. 5 to 7 herein as stand alone discrete entities, 



200207945 

-16- 

each capable of communicating with each other via a wireless link, these three 
components may be integrated into a single physical device, or combinations of 
the components may be integrated into one or more physical devices. Where the 
components are integrated into a single physical device, they may be worn by a 
single host person. 

In an embodiment where the people-observing device and the image 
capture device are both worn by a wearer, the image data may be made 
available directly to the people-observing device via an internal bus or an 
interface. 

Referring to Fig. 8 herein, there is illustrated schematically a set of personal 
sensors worn by a host person for detecting body language exhibited by the host 
person. Sensor types can include the following: 

Motion sensors for detecting the motion of a limb of a person, for example 
raising of an arm, rotation of a head or the like. A motion sensor may be 
implemented in the form of a wrist strap 800 containing a small gyroscope, for 
example a laser gyroscope, or a fibre optic gyroscope. Alternatively, a motion 
sensor may be implemented in the form of a jewelry item, for example an earring 
or ear stud 801 for detecting rapid head rotation. A motion sensor may include 
an accelerometer. Motion sensors may also be worn or attached to other 
clothing items, for example included in the sole of a shoe or other footwear. 

Temperature sensors: Temperature sensors for measuring skin 
temperature or other body temperature may be implemented as jewelry items 
which are worn in contact with the skin, for example as a wrist watch or bracelet. 
Temperature sensors may be included within clothing worn next to a person's 
skin, for example as thin conductive strips woven into a material which a person 
can wear. 
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Voice detectors and sound detectors. A sound detector may be 
implemented within a people-observing device itself, or may be worn externally 
as a headset 802 having a microphone positioned near a person's mouth. 

5 Two-dimensional image sensors: An image capture device itself 803 may 

be used as a sensor. For example the image capture device may capture an 
image of a person other then the host person wearing the image capture device, 
or a general scene external to a host person wearing the image capture device. 

10 There will now be described in further detail, various elements of the image 

capture system. In some cases, the elements described are specific to a 
particular implementation of the image capture system, and it will be appreciated 
by persons skilled in the art that variations in the precise implementation of the 
various elements described may be made. 

15 

Referring to Fig. 9 herein, there is shown schematically an implementation 
of an image capture system comprising 3 portable camera units 900 to 902, each 
said camera unit comprising an image capture device; a people-observing 
device; and an attention detection module, engineered into a single casing which 
2 0 is portable and battery powered, and can be worn, carried or held by a person. 

There will now be described various aspects of operation of the image 
capture system described with reference to Figs. 1-9 herein. 

2 5 Referring to Fig. 10 herein, there is illustrated schematically a still image 

captured by an image capture device, from which an attention clue can be 
abstracted. The image taken is of a person having a facial expression, e.g. a 
smile or laugh. The image capture device captures a 2-dimensional image, 
which is processed by an attention detection module. 

30 

The two dimensional image may be converted into a bitmap, and algorithms 
applied to the bitmap, for edge detection to detect edges of features, and 
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algorithms for detection of human faces or human outlines. These algorithms 
may be resident in either a people-observing device or an attention detection 
module. Facial expressions may be detected, by detecting edges, and then 
comparing a set of detected edges, or alternatively points on those edges, with a 
predetermined library of edges or points which corresponds to a persons smiling 
or laughing facial expression. 

The attention detection module comprises a self perspective attention 
analyzer and an observer perspective attention analyzer. Referring to Fig. 11 
herein, the self perspective and observer perspective attention analyzers may 
share attention detection components, such as common image processing 
components for processing two dimensional images. The image processing 
components include an edge detection algorithm 1101 for detecting edges within 
a two dimensional image; a human outline detector component 1102 for 
detecting a human outline from a two dimensional image frame, or a sequence of 
two dimensional image frames; a facial expression analysis component 1 103 for 
detecting various facial expressions, for example dilated pupils, a smile, a 
surprise expression and like expressions which indicate that the attention of a 
human has been captivated by a subject; a motion analyzer component 1 104 for 
analyzing motion from a sequence of two dimensional images; and a posture 
analyzer component 1105 for analyzing a posture of a human being from a two 
dimensional image or a sequence of two dimensional images. 

Each of the components may be implemented by a general purpose 
computer, under control of an application, or alternatively, may be implemented in 
firmware, for example in an application specific integrated circuit, in which case 
the analyzer component may be manufactured to be small enough to be portable 
and worn upon a host human. 



The attention detection module, in determining whether a persons state of 
attention has been raised sufficiently to activate capture of data on behalf of that 
person, takes into account attention clues collected from a self perspective, and 
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attention clues detected from an observer perspective. However, the attention 
detection module does not necessarily need to give each type of attention clue an 
equal weighting, in determining whether a host persons attention has been 
raised. Depending upon the type of attention clue, and other factors including for 
5 example an environment in which the host person finds themselves, a 
disproportional higher amount of weighting may be given either to self 
perspective attention clues, or observer perspective attention clues. 

Referring to Fig. 12 herein, there are illustrated schematically a set of 
10 sensors feeding instructions to a people observation device. The sensors may 
be incorporated integrally with the people observation device, or may be provided 
externally to the people observation device, and send signals to the people 
observation device. Sensor types include the following: 

is temperature sensors for sensing skin temperature of a host person carrying 

the sensor; 

an image capture device, for example a charge couple delay device for 
capturing images in a self perspective mode of a scene, which may include other 
2 o people; 

a set of biosensors, including a heart rate monitor, a perspiration monitoring 
sensor, a skin conductivity monitoring sensor; 

2 5 one or a plurality of motion sensors for sensing parameters such as head 

rotation, arm movement and leg movement; 

one or more audio sensors, for example a microphone. 

3 0 Each of the sensors feed sensor signals to the people observation device. 

The people observation device processes the sensor signals, into a form into 
which they can be analyzed by the attention detection module. 
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The agent device of the image capture system may be operated in various 
modes of operation including a self perspective mode, or an observer perspective 
mode. Each of these modes of operation require a different configuration of the 
components of the system. 

The modes function as follows: 

In the self perspective mode of operation, attention of a person is activated 
from the perspective of a host person carrying an image capture device. The 
host person may become interested in various aspects of their environment. For 
example, the person may look at an airoplane in the sky, or may turn around to 
look at a passing person, or may have their attention attracted by an event, for 
example a load bang, a car crash or some other distracting event in the persons 
immediate surroundings. Sensors carried by the person monitor the host 
person's body language. A people-observing device carried by the host person 
receives signals from the sensors, and generates attention clue signals. The 
attention clue signals comprise signals from the sensors at various levels of data 
processing. For example in the case of a two dimensional image data, this may 
be processed very little, and may undergo almost no processing of the 
information content of the signal prior to being forwarded to the attention 
detection module. In the case of an audio signal, the information in the audio 
signal may be processed to before being presented to the attention detection 
module. For example the audio signal may be filtered such that only audio 
signals above predetermined amplitude are passed to the attention detection 
module, with normal background environment signals at a relatively low level of 
loudness being filtered out by the people-observing device. The level of 
information processing to be applied to the sensor signals before passing those 
signals on to the attention detection module is an implementation specific design 
parameter which may be varied from system to system. 
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In the self perspective mode, the attention detection module receives 
signals from the host wearer, which represent the body language of the host 
person, to which the attention detection module is assigned to monitor, that is, 
represents, information describing an attention level from the perspective of the 
5 host person. These self perspective attention clues are analyzed, and as a result 
of the analysis, an image capture signal for activating an image capture device 
worn by the host user is generated when the analyzer determines that the 
attention of the host user is raised to such a level that there is a significant or 
interesting event in the host users immediate environment, of which an image 
10 should be captured. 

In an observer perspective mode, one or more people-observing devices 
are used to observe persons other than a host person wearing the people- 
observing device. For example where first and second host persons each carry 

15 first and second corresponding respective people-observing devices, and each 
carry a corresponding attention detection module, a first people-observing device 
of a first host person may observe the second person. Observation can take one 
of the forms mentioned above, for example continuous monitoring of a moving 
captured image of the second person. Upon detecting a significant sensed 

2 0 parameter, the first people-observing device generates an attention clue signal 
which may be processed by the first attention detection module carried by the 
host person. Attention clues relate to the second person, rather than the first 
host person. 

25 The first attention detection module, having determined that the second 

person's level of attention is raised to a high level may send signals to the first 
image capture device worn by the first person, and/or an activation signal to the 
second image capture device worn by the second person for either one or both of 
those devices to capture a corresponding respective image. 

30 

Conversely, the second people-observing device worn by the second 
person monitors the attention of the first person or any other third person in the 
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environment. The second people-observing device, having observed behavior of 
the first person, may generate attention clues which it sends to the second 
attention detection module. The second attention detection module analyses 
those attention clues and where appropriate, generates one or more trigger 
signals and sends those to the second image capture device and/or first image 
capture device to trigger capturing of an image or image sequence by the second 
and/or first image capture devices. 

Referring to Figure 13 herein, there is illustrated schematically interaction 
between first and second attention detection modules 1300, 1301 respectively 
where first attention detection module 1300 is worn by a first host person, and 
second attention detection module 1301 is worn by a second host person. 

First attention detection module performs attention detection processes 1, 
2, 4, 10 based upon a plurality of attention clue signals, 1, 2, 4, 10 respectively. 
The first attention detection module receives data from the second attention 
detection module which is collected in an observer perspective mode by the 
second attention detection module. The second attention detection module 
monitors the first host person, and upon detecting an attention clue of the first 
host person, signals to the first attention detection monitor information concerning 
the first host person, which is analyzed by the self perspective analyzer of the first 

c 

attention detection module 1300. 

Conversely, first attention detection module 1300 monitors the second 
person, and upon detecting an attention clue of the second person signals to the 
second attention detection monitor 1 301 attention clues concerning the second 
person. The second attention detection monitor 1301 incorporates the 
information received from the first attention detection monitor, in a self 
perspective analysis mode of the second attention detection monitor, in which the 
second detection monitor analyses attention clues of the second host person in 
order to determine whether an image signal should be generated for capturing an 
image using the second persons image capture device. 
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Claims: 

1 . An attention detection system comprising: 

5 

at least one sensor device capable of generating signals of a host person 
from a self-perspective and relating to attention clue signals; and 

at least one sensor device capable of generating signals of a host person 
10 from an observer perspective and relating to said attention clue signals; 

a portable attention detection module capable of receiving the self 
perspective and observer perspective attention clue signals and which is capable 
of determining a situation of raised attention of said host wearer from said self 
15 perspective attention clues, and said received observer perspective attention 
clues. 

2. The attention detection system as claimed in claim 1, further 
comprising: 

' 20 

an image capture device for capturing an image from a self perspective of 
said host wearer, in response to a determined situation of raised attention. 

3. The system as claimed in any one of the preceding claims, wherein 

2 5 said at least one sensor device for generating signals from a host perspective are 

adapted for wearing by said host person. 

4. The system as claimed in any one of the preceding claims, wherein 
said at least one sensor device for generating signals from an observer 

3 o perspective is adapted to be worn by a further person. 
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5. The system as claimed in any one of claims 1 to 3, wherein said at 
least one sensor device for generating signals from an observer perspective is 
adapted to be placed in an environment which said host person may occupy. 

6. The system as claimed in any one of the preceding claims, further 
comprising a people-observing device, which is configured for communicating 
with said attention detection module. 

7. The system as claimed in claim 6, comprising a plurality of people- 
observing devices, which are capable of: 

communicating with said attention detection module; and 
communicating with each other. 

8. The system as claimed in claim 6 or 7, comprising at least one 
camera device. 

9. The system as claimed in any one of the preceding claims 
comprising first and second people-observing devices adapted for using beacons 
to locate and detect each other. 

10. The system as claimed in any one of the preceding claims which is 
integrated into a host wearable device. 

11. The system as claimed in any one of the preceding claims, 
comprising a people-observing device capable of communicating with said 
attention detection module, wherein said people-observing device is configurable 
for cooperating with at least on other people-observing device for communicating 
information with said at least one other people- observing device. 
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12 The system as claimed in any one of the preceding claims, 
comprising a people-observing device capable of communicating with said 
attention detection module, wherein said people-observing device is configurable 
for recognizing ate least one other people-observing device to form a group of 
5 people-observing devices which recognize each other. 

13. The system as claimed in claim 12, wherein said people-observing 
device is configurable for being set to communicate or not communicate with at 
least one other people-observing devices based on an analysis of contextual 

10 information relating to a host person. 

14. The system as claimed in claim 1, comprising a people-observing 
device configurable for being set to communicate or not communicate with at 
least one other people-observing devices based on an analysis of a type of 

15 attention clue detected. 

15. The system as claimed in claim 1, comprising at least one people 
observing-device configurable for being set to communicate or not communicate 
with at least one said attention detection module depending upon a type of 

2 o attention clue detected. 

16. The system as claimed in claim 1, comprising a digital camera 
device capable of capturing a 2-dimensional digital image. 

25 17. The system as claimed in any one of the preceding claims, wherein 

the sensor device detects a facial expression of a said person. 

18. The system as claimed in any one of the preceding claims, wherein 
the sensor device detects an eye direction of a said person. 

30 

19. The system as claimed in any one of the preceding claims, wherein 
the sensor device detects an attitude of movement of a said person. 
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20. The system as claimed in any one of the preceding claims, 
comprising a means for detecting a body posture of a said person. 

21. The system as claimed in any one of the preceding claims, 
comprising means for detecting a context of an environment containing a said 
person. 

22. The system as claimed in any one of the preceding claims detects 
a voice utterance of a said person. 

23. A method of capturing images using at least one camera device, 
said method comprising: 

detecting an attention clue exhibited by at least one human person, from a 
perspective of a host person carrying said at least one camera device, said 
attention clue indicating that said persons attention is drawn by a subject; and 

detecting an attention clue of said host person from an observer perspective 
external of said host wearer; 

activating said at least one camera device for capture of an image of said 
subject, in response to detection of said self perspective attention clue and said 
observer perspective attention clue. 

24. The method as claimed in claim in claim 23, wherein a said step of 
detecting an attention clue comprises detecting a facial expression of said at 
least one person. 

25. The method as claimed in claim in claim 23, wherein a said step of 
detecting an attention clue comprises detecting an eye direction of said at least 
one person. 
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26. The method as claimed in any of claims 23 to 25, wherein a said 
step of detecting an attention clue comprises detecting an attitude of movement 
of said at least one person. 

5 

27. The method as claimed in claim in any one of claims 23 to 26, 
wherein said step of detecting an attention clue comprises detecting a body 
posture of said at least one person. 

io 28. The method as claimed in claim in any one of claims 23 to 27, 

wherein said step of detecting an attention clue comprises detecting a voice 
utterance of said at least one person. 

29. The method as claimed in any one of claims 23 to 28, wherein 
15 detecting an attention clue comprises: 

capturing a two dimensional image; and 

applying an image processing to said two dimensional image to detect an 
2 o attention clue selected from the set: 

a facial expression; 

an eye direction; 

25 

a body movement; 
a body posture. 

30 30. A method of automatically capturing an image, said method 

comprising; 



200207945 

-28- 

detecting at least one attention signal in response to a detectable body 
parameter of at least one human person; 

analyzing said at least one attention signal to determine an interest level of 
said at least one human person; and 

activating capture of a said image, in response to said interest level. 

31 . The method as claimed in claim 30, comprising; 

determining a situational saliency of a scene, by analyzing said at least one 
attention signal. 

32. The method as claimed in claim 31 or 32, wherein said analysis is 
performed in a mode of self perspective of a said person. 

33. A method as claimed in claim 31 or 32, wherein said analysis is 
performed in a mode of an observer perspective of said at least one person. 

34. An image capture device comprising: 

an image detector device capable of capturing a two dimensional image; 

an attention detection component for determining an attention signal of a 
person from a self perspective; 

a transponder device for receiving activation signals from a remote source; 

wherein said attention detection component is configured for identifying 
said activation signals, and activating capture of an image in response to a said 
self perspective activation signal and said received activation signal; 
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35. A people observing device for observing at least one person, said 
people- observing device comprising: 

an interface for interfacing with at least one sensor device; and 

means for receiving sensor signals, said sensor signals representing 
aspects of a persons body language, which are observed from a position external 
of said person; and 

an analyzer for determining from said sensor signal signals at least one 
attention clue of said observer person; and a transmitter for transmitting attention 
signals. 

36. The people monitoring device as claimed in claim 35, further 
comprising: 

a transponder device capable of transmitting said sensor signals. 

37. The people monitoring device as claimed in claim 35 or 36, further 
comprising: 

an image capture device for capturing two dimensional image frames. 

38. An attention detection component for determining a level of 
attention of at least one person, said component comprising: 

analyzer means for analyzing at least one attention clue signal, and 
determining from said attention clue signal, a level of interest of said at least one 
person. 

39. The attention detection component as claimed in claim 38, 
operable for: 
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analyzing said attention clues in a self perspective mode, in which said 
attention clues relate to a single host person. 

5 40. The attention detection component as claimed in claim 38, 

operable in an observer perspective mode, in which said attention clues 
represent signals describing behavior of a person observed from a remote 
location. 

10 41. The attention detection component as claimed in claim 38, 

comprising a transponder device for receiving said attention clue signals from a 
remote sender device. 



42. Computer program product comprising: 

15 

an analysis component for analyzing a plurality of sensor signals 
representing attention clues collected from a self perspective of a person, and 
attention clues collected from an observed perspective of said person, and 
determining from said sensor signals, a behavioral mode of a person; and 

20 

a component for generating an image capture trigger signal for triggering an 
image capture device to capture a two-dimensional image data, in response to a 
said sensed behavioral mode of said person. 

2 5 43 An attention detection system comprising: 

a portable attention detection module capable of receiving attention clues 
generated from a self perspective of a host wearer of said attention detection 
module; 



30 
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a people-observing device capable of observing said host wearer from an 
observer perspective external of said host wearer and determining attention clues 
of said host wearer from said observer perspective externally of said host wearer; 

wherein said attention detection module is capable of determining a 
situation of raised attention of said host wearer from said self perspective 
attention clues, and said received observer perspective attention clues. 
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Abstract 

ATTENTION DETECTION 

5 An attention detection system comprises: a portable attention detection module 
capable of receiving attention clues of a host person, said attention clues 
generated from a self perspective of said host person, and receiving attention 
clues of said host person generated from an observer perspective; at least one 
sensor device capable of generating signals relating to said attention clue signals 
10 of said self perspective; and at least one sensor device capable of generating 
signals relating to said attention clue signals of said observer perspective.; 
wherein said attention detection module is capable of determining a situation of 
raised attention of said host wearer from said self perspective attention clues, 
and said received observer perspective attention clues. 

15 
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